Bound Controller for a Quadruped Robot using Pre-Fitting Deep Reinforcement Learning

The bound gait is an important gait in quadruped robot locomotion. It can be used to cross obstacles and often serves as transition mode between trot and gallop. However, because of the complexity of the models, the bound gait built by the conventional control method is often unnatural and slow to compute. In the present work, we introduce a method to achieve the bound gait based on model-free pre-fit deep reinforcement learning (PF-DRL). We first constructed a net with the same structure as an actor net in the PPO2 and pre-fit it using the data collected from a robot using conventional model-based controller. Next, the trained weights are transferred into the PPO2 and be optimized further. Moreover, target on the symmetrical and periodic characteristic during bounding, we designed a reward function based on contact points. We also used feature engineering to improve the input features of the DRL model and improve performance on flat ground. Finally, we trained the bound controller in simulation and successfully deployed it on the Jueying Mini robot. It performs better than the conventional method with higher computational efficiency and more stable center-of-mass height in our experiments.Comment: 7page

Multi-expert learning of adaptive legged locomotion

Achieving versatile robot locomotion requires motor skills which can adapt to previously unseen situations. We propose a Multi-Expert Learning Architecture (MELA) that learns to generate adaptive skills from a group of representative expert skills. During training, MELA is first initialised by a distinct set of pre-trained experts, each in a separate deep neural network (DNN). Then by learning the combination of these DNNs using a Gating Neural Network (GNN), MELA can acquire more specialised experts and transitional skills across various locomotion modes. During runtime, MELA constantly blends multiple DNNs and dynamically synthesises a new DNN to produce adaptive behaviours in response to changing situations. This approach leverages the advantages of trained expert skills and the fast online synthesis of adaptive policies to generate responsive motor skills during the changing tasks. Using a unified MELA framework, we demonstrated successful multi-skill locomotion on a real quadruped robot that performed coherent trotting, steering, and fall recovery autonomously, and showed the merit of multi-expert learning generating behaviours which can adapt to unseen scenarios

Multiagent Consensus Control under Network-Induced Constraints

Mean consensus problem is studied using a class of discrete time multiagent systems in which information exchange is subjected to some network-induced constraints. These constraints include package dropout, time delay, and package disorder. Using Markov jump system method, the necessary and sufficient condition of mean square consensus is obtained and a design procedure is presented such that multiagent systems reach mean square consensus

Average Consensus Analysis of Distributed Inference with Uncertain Markovian Transition Probability

The average consensus problem of distributed inference in a wireless sensor network under Markovian communication topology of uncertain transition probability is studied. A sufficient condition for average consensus of linear distributed inference algorithm is presented. Based on linear matrix inequalities and numerical optimization, a design method of fast distributed inference is provided

Cortico-hippocampal computational modeling using quantum neural networks to simulate classical conditioning paradigms

Most existing cortico-hippocampal computational models use different artificial neural network topologies. These conventional approaches, which simulate various biological paradigms, can get slow training and inadequate conditioned responses for two reasons: increases in the number of conditioned stimuli and in the complexity of the simulated biological paradigms in different phases. In this paper, a cortico-hippocampal computational quantum (CHCQ) model is proposed for modeling intact and lesioned systems. The CHCQ model is the first computational model that uses the quantum neural networks for simulating the biological paradigms. The model consists of two entangled quantum neural networks: an adaptive single-layer feedforward quantum neural network and an autoencoder quantum neural network. The CHCQ model adaptively updates all the weights of its quantum neural networks using quantum instar, outstar, and Widrow–Hoff learning algorithms. Our model successfully simulated several biological processes and maintained the output-conditioned responses quickly and efficiently. Moreover, the results were consistent with prior biological studies

Cortico-hippocampal computational modeling using quantum-inspired neural networks

Many current computational models that aim to simulate cortical and hippocampal modules of the brain depend on artificial neural networks. However, such classical or even deep neural networks are very slow, sometimes taking thousands of trials to obtain the final response with a considerable amount of error. The need for a large number of trials at learning and the inaccurate output responses are due to the complexity of the input cue and the biological processes being simulated. This article proposes a computational model for an intact and a lesioned cortico-hippocampal system using quantum-inspired neural networks. This cortico-hippocampal computational quantum-inspired (CHCQI) model simulates cortical and hippocampal modules by using adaptively updated neural networks entangled with quantum circuits. The proposed model is used to simulate various classical conditioning tasks related to biological processes. The output of the simulated tasks yielded the desired responses quickly and efficiently compared with other computational models, including the recently published Green model

Multiview Active Learning for Scene Classification with High-Level Semantic-Based Hypothesis Generation

Multiview active learning (MVAL) is a technique which can result in a large decrease in the size of the version space than traditional active learning and has great potential applications in large-scale data analysis. This paper made research on MVAL-based scene classification for helping the computer accurately understand diverse and complex environments macroscopically, which has been widely used in many fields such as image retrieval and autonomous driving. The main contribution of this paper is that different high-level image semantics are used for replacing the traditional low-level features to generate more independent and diverse hypotheses in MVAL. First, our algorithm uses different object detectors to achieve local object responses in the scenes. Furthermore, we design a cascaded online LDA model for mining the theme semantic of an image. The experimental results demonstrate that our proposed theme modeling strategy fits the large-scale data learning, and our MVAL algorithm with both high-level semantic views can achieve significant improvement in the scene classification than traditional active learning-based algorithms

Impedance Control and its Effects on a Humanoid Robot Playing Table Tennis

This paper proposes an impedance control scheme used on humanoid robots for stability maintenance when the robot is expected to carry out fast manipulatory tasks. We take table tennis playing as an example to study this issue. The fast acceleration required by table tennis rallying will result in an unknown large reaction force on the robot, causing the body to swing back and forth in an oscillating motion and the foot to lose complete contact with the ground. To improve the stability during fast manipulation and in order to resist disturbances due to the reaction force, we introduce impedance control to absorb the impact and decrease the amplitude of body swinging. The system's adjusting time is also reduced and the oscillations are eliminated according to the experimental results, which show the effectiveness of our scheme

Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics

Locomotion control has long been vital to legged robots. Agile locomotion can be implemented through either model-based controller or reinforcement learning. It is proven that robust controllers can be obtained through model-based methods and learning-based policies have advantages in generalization. This paper proposed a hybrid framework of locomotion controller that combines deep reinforcement learning and simple heuristic policy and assigns them to different activation phases, which provides guidance for adaptive training without producing conflicts between heuristic knowledge and learned policies. The training in simulation follows a step-by-step stochastic curriculum to guarantee success. Domain randomization during training and assistive extra feedback loops on real robot are also adopted to smooth the transition to the real world. Comparison experiments are carried out on both simulated and real Wukong-IV humanoid robots, and the proposed hybrid approach matches the canonical end-to-end approaches with higher rate of success, faster converging speed, and 60% less tracking error in velocity tracking tasks